;ume*j 



DOCUMENT RESUME 



ED 214 161 

J 

AUTHOR * 
.TITLE * 

INSTITUTION 



CS 006 594, 



SPONS AGENCY 
REPORT NO 
PUB DATE 
CONTRACT 
NOTE 



Polin, Ruth M. . ' 

A Study of Preceptor-Training of Classroom Teachers 
* in Reading Diagnosis. .1 

Michigan stated Univ. , East .Lansing. Inst , for 1__ 

Research on -Teaching. 

'National, liist. of Education (ED), Washington, D.C. 
iHt-RS-110 . . < 



Nov 81 • . 
; 400-81-0014 ' • . t * 

48p. ; Several pages may be marginally legible. .>/ 
AVAILABLE FROM The Institute for Research on Teaching, College of. 

.« Education, Michigan state University, 252 Erickson 
- ♦ , . Hall, East Lansing, MI 48824 ($3.50). 



EDRS PRICE 
DESCRIPTORS 



ABSTRACT 



MF01/PC02 Plus Postage.' * 
♦Elementary School Teachers'; Inservice Teacher v 
Education; Learning Problems'; ^Reading Diagnosis; 
Reading Difficulties; *Reading Research; *Skill 
Deyel<^ment • „*Trainers; „..*Training Methods 



pf small group 



A atudy investigated the* effects of differing type 
instruction on the diagnostic performance of classi 



types 

* — — c - « — - - — w *. • w«« vt# wiuvjiivj o i. uauwe i*i.assroom 

t6achers f wljo were inexperienced in reading diagnosis. Fifteen 
elenpentary school teachers were divide^ injto three groups and traifted 
for 40 hours by three different preceptors using differing approaches 
to thje Same model of heading/ Each group gained experience ih dealing 
with/either real cases, simulated cases, or both, with instructor" 
feedback, and eaqh was instructed the use of decision aidjs to 
guide /fnteractiop in the simulated cases. Student progress was , • 
ponitored by means of & pretest , ;midtest, and^posttest on a simulated 
,jase, and an additional, posttest on a. c^se not previously diagnosed. 
The students diagnosed" the* cases. usU^a structured*, form and 
transferred- their diagnoses to v a cij£li,s\t. Statistics wa^e then run 

3Pf hi* 



, on agreement between each student 



is or her prteceptorV agreement 



— — -r^ w . tr *<» ^ 

among the students, and ^agreement atyong the preceptors. Results 
showed that student agreement with preceptor and wilh other students 
increased in jail aspects of a diagnosis; .however , transfer of skills 
to a new case was questionable/ Appendixes contain copies of .the 
subject instructions, for training sessions A * a dia<gftostic checklist / 
and an explanation of the statistical analysis used in the study, 
(FLK f . * « 



.- j 

* J Reproductions supplied by EDRS are the b^st 'that 'can be made- * 

* * from the original document. " * * 



O ■ 

ERIC , 



DOCUMENT RESUME 



ED 214 161 

AUTHOR - 
.TITLE * 

INSTITUTION ' 

SPONS AGENCY 
REPORT NO 
PUB DATE 
CONTRACT . 
NOTE 

AVAILABLE FROM 



CS 006 594, 



EDRS PRICE 
DESCRIPTORS 



Polin, Ruth M. . ' 

A Study of Preceptor? -Training of Classroom Teachers 
in Reading Diagnosis. 

Michigan State Univ., East Lansing. Inst, for il. 

Research on -Teaching. 

National, Inst, of Education (ED), Washington, D.C. 
IRT-RS-110 . ^ 

Nov 81 

400-81-0014 " * - L . . f 

48p.; Several pages may be marginally legible. 
The Institute for Research on Teaching, College of. 
Education, Michigan State University, < 252 Erickson 
Hall, East Lansing, MI 48824 {$'3. 50). 

MF01/PC02 Plus Postage.' • 
♦Elementary School Teachers'; Inservice Teacher v 
Education; Learning Problems; *Reading Diagnosis; 
Reading Difficulties; *Reading Research; *Skill 
Develd^pmenti ^Trainers; ^Training Methods 



ABSTRACT " ' 

, A atudy investigated the* effects of differing types 

pf small group instruction on the diagnostic performance of classroom 
t6achers f wljo were inexperienced in reading diagnosis. Fifteen 
eleijrentary school teachers were divided into three groups and traifted 
for 40 hours by three different preceptors using differing approaches 
to thje Same model- of .reading/ Each group gained experience ih dealing 
with/either real cases, simulated cases, or both, with instructor 
feedback, and eaqh was instructed in" the use of decision^ aids to 
guide /fnteractiop in the simulated cases. Student progress was , • 
-oftitored Jby means of i pretest , *midtest, and^posttest on a simulated 

ase, and an additional, posttest on a^c^se not previously diagnosed. 
The students diagnosed' the* cases. uS^cf,a structured form and 
transferred their diagnoses to„a cJ^a^isvt. Statistics ware then run 
on agreement between each student 5Pq his or her preceptor^ agreement 
among the students, and ^agreement a^ong the preceptors. Results 
showed that student agreement with preceptor and wi%h other students 
increased in jail aspects of a diagnosis; .however, transfer of skills 
to a~ new case was questionable/ Appendixes contain copies of .the 
subject instructions, for training- sessions,' a dia<gftostic- checklist / - 
and an explanation of the statistical analysis used in the study. 
(FLK * • * § • • 



J 

***************************** ****************************************** 

* J Reproductions supplied by EDRS are the b&st 'that 'can be maiie. * 

* * ' * from the original document. ** * 
***************************** ****************************************** 



. ^ x "^A«)NAlMsou».C£8 .FORMATION / 
CENTER (ERIC) ' 

■ ' l 00 ^"' h3s ^ 'to***** » 

ong f r^| f ng it « 

M,oo» change h, 3v * been made to ,mp,ove 
'eprcxiyctton q u9 f tIy r 

_ • v not neceswmy ,ep«s*nt offtoaifMlE 

pOS/tfOn or poky 



Research Seties No. 110 



A STUI)Y OF PRECEPTOR TRAINING, 
OF CLASSROOM TEACHERS - 
IN READING DIAGNOSIS 



Ruth M. Polin 




\ 



•V 



•> - Published By 

. The Institute for Research on Teaching 
252 Erickso^ Hall / 
Michigan State University 
East Lansing, Michigan 48824 
t 



November 1981 



This work is sponsored in part by the Institute for Research on 
^Teaching., College of Education, "Michigan. State University. The Institute 
for* Research on Teaching is funded primarily by the Program for Teaching* 
and Instruction of the National Instate of "Education, United Staters 
Department of Education. The opinions expressed in this publication, 
do not necessarily reflect the position, policy, or endorsement of the 
National ^Institute of -Education, (Contract No, 400-81-0014) * 

* » . J * „ & -< ' 



'*/ * .Institute. for Research on Teaching' ■• ■ "' 

- . — - — ~ ... - . ■ '*-\ 

. ^ Teachers' thoughts and decisions are the focus of-, studies" cur ijentfl^ " 
under W ay ( at Michigan State' University's Institute f^H^eych . on Teach- • , * 
ing (IRT). The IRT was ^founded in April 1976with a $3.6-million grant from 
. the- National institute of Education. A*new grant obtained in" 198i from the - 
NIE extends the IRT's work' through 1984. Funding is also received" from \ ' 
other agencies' and foundations. The- -ihstritute, has major projects investigating 
teacher decisionmaking, including studies pf reading diagnoses ^nd radiation 
classroom management strategies, instruction in the 4r.eas of language arts', ' 
reading, and mathematics, Wcnef education, teacher "planning, affedts of < * • 
external pressures' on .teachers' decisions; socio-cultu4l factors, and . 
teachers ' perceptions of . student affect.. Researchers from many different ' 
disciplines cooperate in IRT research. In addition, public ' school' teachers-' - 
work at IRT as half-time collaborators in research, helping to design -and 
plan studies, collect da.ta, and analyze results. Th &; Institute .publishes^ 
research reports, cinference proceedings, occasional papers, and a free' ' 
quarterly newsletter for. practitioners. For more information or to be placed 
on the IRT mailing list, please write to.': The IRT Edi'tor* 252 Erickson, MSU, 
East Lansing, Michigan 48824. , • , - 



Co-Directors.: Jeire E. 'Brophy and Andrew C. Porter 
Associate Directors:" Judith E. Lanier and Lee S'. Shulman 
Editorial Staff,: . - 

' - * * 

.Janfet £legg, IRT ed'itoc 1 1 
*Pat luschan, assistant editor 



A STUD.Y OF PRECEPTOR TRAINING OF CLASSROOM TEACHERS ' .- 

r IN READING "DIAGNOSIS 1 



4 



Ruth M. Polin 2 " 



Clinician agreement on diagnosis, presumed necessary for effec- 

s . * 4 * s 

tive remediation of children's reading^, difficulties, has been the . 
focus of in-depth investigation by the IRT's Clinical Studies* Project 
f<3r a number of years. Xhe theoretical base for the project' is a * 
theory of clinical problem-solving behavior by El'stein ^nd Shultaan 1 
<Elstein, Shulman, & Spratka, 1978), which was refprmula'ted, for educ- 
tion* by Vinsonhaler and Wagner— (Vinspnhaler, Wagner,'* & .Elstein," 1977), 

This* theory postulates that problem-salving behavior is "a function * 

j - ■ ^ 

qf clinicians' memories and strategies, which influence the decision- 

V. 

making process as clinicians diagnose cases, Memories <and strategies 

are determined largely by the training clinicians or teachers receive.* 

■* t 

Clinical decision making in reading has b^en investigated with* 
several classifications of professionals (reading clinicians, classroom 
teachers, learning disabilities socialists, and school psychologists) 
in" a series of , observational studies from 1977 to #980, and the results; 
of tftese studiee^have bee^ presented by Weinshan^c (Note 1). There was 
low diagnostic agreement among all professional groups except sfchopl 



1 ' w 

A comprehensive paper, on all the training ptudies will, be ^vailabl 

at a later date. . * " „ " # * 

* • • 

2* . ' ' • . 

Ruth Politi is data-processing coordinator for t>er ^Clinical Studies 

Project. i * * ' 

* < "»•'** - 

The author acknowledge^ the contribution of the following members 
of the Clinical Studies Project andsconsultants who assisted in the' 
^planning^ development of. Computer systretys fofc data agal^sis, .and 
operation of the study; John F. Vinsbnhalery Christian 'Wagner, % % ^ 
AnnettP Weinshank, George Sherman, Gloria Blatt, Doron (|fl, Deborah 
Cureton, Linda Vavrus, and Gwyheth Stewart. % * , * . 4 V ' * 



psychologists, whose trailing and experience differs markedly from ' 
that o£ the others mentioned** \ % 

r * <m 

To explore the Instructional Corollary to thi« Inquiry Theory, 
which -states th&t clinical performances (e.g., reading diagnosis agree- 
ment), can &e* improved by improvements in clinical memory and strategy, 
a series of training (application) studies were undertaken. Simply * 
staged., what a clinician tetaihs from his/heif educational background 
about how a child leaifns to r^acl and what, procedures should.be followed 
in assessing the child's mastefy .of the fundamental" elements of reading 
constitute the clinician's memory and. strategy. More~ intensive, organized 
training and clinical experience should thus improve the diagnostic 

' decision -making of these clinicians an<}, ultimately, the remediation of 
* 

a child'* s reading disability. 

The first of these application studies was conducted in 1977 with 
the students of the graduate heading diagnosis course at Michigan State ~ 

o ■ t , 

University {Sherman, Weinshank*, & Brown, Note 2). In 1979, another 

« S * 

X 

, study was conducted in the same setting, >but*with the additioji^l use of 
* < 

a model of reading for instruction and of 'decision aids (diagnostic 

write-up forms and checklists). Simulated casfes developed by the 

Clinical Studies Project were used in all studies. Simulated cases 

are simply collections of information about children with reading problems 

They are based on real -children in grades three through nine who 

attended the Michigan State University Reading Clinic and whose reading 

problems were Considered to be representative of those frequently noted 

in public schools* 'A detailed description of these cases is presented 

by Weinshank (Note 1), /The 1979 study* Showed the efficacy of using 

diagnostic ^decision aids in channeling a clinician's thinking along ^ 

« 

^specified, orderly lines; this resulted in superior inter-clinician 



' agreement (£il, Poling Vinsonhaler, &.,VanRoekel, Note 3). 

That result prompted further investigation of the effect of training 
on diagnostic agireetaent. ,Jn*1980, the study, reported here was can- 
ducted. In this study,* I and my colleagues on' the Clinical Studies 
Project investigated the effect of differing- types of small group 

instruction on the diagnostic .performance of classroom teachers who 

. - /' v 

were inexperienced in reading diagnosis. 

Objectives " , 

The objectives of this application study were to attempt to answer* 

4 . A * J 

* * w <r 

the following questions. . 

1. Sirice the 1979 application study showed* the positive 
effect of systematized [trailing* in reading diagnosis, 
would different contetfF and training methods al^o 
have a positive effect? 
% • * * " 

I 2*. Does- practice with simulated cases have a greater / 
, effec £ * n diagnostic performance than the necessarily 
limits !pVactice with\real children? * ' ^ 

* i * 

• ,3. If' limited^ training results in higher diagnostic £ v gree- ■ 
, jnent, can even higher agreement be expected with m#re 
exCensixe training? 

) • Methods ( r ' 

. Subjects* ' , , , . * 

Ten classroom teachers' with litfcle or no training or experience* 

in reading d'iagnpsis, drawn f rom ^elementary ^schools of the Lansing* 

Michigan school district^ and five students enrolled in the Michigan 

* * ■ * 

State University graduate*- levgl course in reading diagnosis, also 

with mipimal experience in reading diagnosis, participated^ in the study. 

The students _ wer^ divided *into* three preceptor training groups, each 

with a'^tiffferent instructor ,(pr£eeptor) , Each, instructor had a somewhat 
\ " _ . ' • , ^ > " V 

' different approach to v teaching 'reading diagnosis. However, all groups 



were instructed fqr a minimum of 30 hours plus 10 hour's of extra prac- 
tice time with (1) th,e Model of Reading and Learning to Read <M0RAL) , ^ , 
a* systematizes cause and effect approach to reading difficulty diagnosis 

(Sherman,. Note 4); (*2) either simulated cases,' real cases, qr both, . ' 

^1 \ / ■ i 

with instructor feedback; and ft) decision aids , ; wtffEtf* guided thfe 

interaction of*thpse using the simulated cases. Progress was monitore^ 

by means of pre-, midr, and post tests on a simulated case, and an * 

if - 
additional posttest on a case not previously diagnosed. Five simulated 

cases were usecj; one subject frojA each predeptor training grou£> was 

tested on each case. - , ' 

Instructional Design \ 1 

The vital signs of heading are the b^sis for group instruction 

y 

in diagnosis of casas of reading difficulty. These vital sign^ indicate 

- * ' \ * 

the reader r s "healfcfi," much like medical vital signs indicate' a 

patient.' s health. They are as follows: 

1. Instant Word Recognition — the ability to' recognize words 

• without hesitation. - It is measured by size of sigh^word 
vocabulary, relative to a child's gr^d^ placement in school. 

* * * ° 

2. Deeoded Word Recognition—the .ability to recognize un- 
familiar wo'rds through use of graphemic similarities. 

3. Meaning^ Vocabulary-?- the scope of words that denote meaning- 
ful relationships to the reader. < 

A. Orai Rehdinfy — the ability to read aloud with fluency and 
inflection. - i * 

f *~ / 

5. Reading Comprehension — the ability to understand and put r 
into meanirigful perspective material that is read. e ^> 

♦ rl 6.* Listening Comprehension — the ability to understand and pUt . 
• .into meaningful perspective material that is heard. 



7. Attention/ Motivation — the ability to activate and maintain 



concentration on<the task at hand. ^- 
The preceptor 'for Gi&up 1 used four vital signs. He merged * 
Vital Signs 3 and 4, calling them Fluent Text Segmentation, and 5" and 



9 t 

6, calling ,them Retentive Comprehension. He considered Vital Sign 7 

( . * 4 * 

a learning effecting factor, rather thai!, a vi£al sign^ (A^learn^ng 
•effecting factor is something that directly influences learning.) ; 
Group. 1- stressed diagnosis according to pe"r?o£mance level on the 



N 



four vital 'signs; the other two groups used all seven as separate 



vital signs. 



Classroom Instruction 

1 ' * ' 

For Groups 2 and, 3, the formal 1 classroom" instruction in reading" 

/ * 
diagnosis was conducted in three-hour blocks weekly with an additional 

m 

lh hours per week spent in diagnosing computer-based simulated cases 

^ (as opposed to the manually-based ones used for the* -test sessions)., 

Computer-based simulated cases contained the same information as 

the manual simulated cases' but we're used in conjunction with a 

computer terminal', which had test information stored on di§ks. This 

information could be readily reprieved for instant display without the 

use of, the cymhersome file box (manual).' Any material that could not 

bz storfed on the disks (fc.g,, audio recordings, actual test booklets^ 

were contained in a loose-leaf study guide. After interacting' with 

a simulated v case, vstudemts\ filled^ out the decision-aid diagnosis 

sheets. Each student then^transla&Sd his/her diagnosis to a standar- 

dized "checklist, indicating whether the case showed adequacies or 

inadequacies in the seven vital signs ani their effecting factbps ^ 

as postulated by the Model of Reading and Learning to Read. (This 

procedure i^ described in detail in the next section of this papeV.) 

Students in Group 1, who used real caseelyither than the simulated 

ones, did not use the checklist sf or theflfcasea. 'instead, the pre- 

ceptor analyzed in class the real cases diagnosed by each student, 
* ♦ i 

and this in-class analysis provided practice comparable to that of 



/ • • - ■ - • , - 

the other two^ groups. Some of the instructional differences across 
the three training groups are summarized in Table 1. 

Ihe diagnosis and remediation chart used in Groups 2 and 3 is 

shown, for one % vital kigijf. in'Figure 1. The same format was used 

for each oiN*the seven vital signs. Group l ! s flowchart also outlined 
steps* to be followed in diagnosing casks of read ing # difficulty 
according to vital signs and effecting ^factors. # 



est Sessions u 



At each test session, subjects were presented with a manually-based 

simulated case. The students received* written and oral instructions > 

on how to, find the information knd how "to complete the test session 
• • ►* 

(see Appendix A). v , \? 

After receiving instructions-, . subjects* observed thfe initial m » % 
• contact information about the case, 'which included a short, summary' 
about the child's reading performance. The subjects were then given 
45 minutes to collect ^as many. cues (items of information) about the 
' »case as they wished. These cues were later recorded as part of' thfeir 
data fc/r the test. , . / 

At the end of 45 minutes, £ach student was asked to write a 
diagnosis of the case, using the decision aid, a sataple of which is 
shown in Figure 2, based on the child's adequacies or inadequacies on 
the seven vital .signs. The students w^ere giaren ^30 minutes to do this. 

After completing their diagnostic w,rite-ups, students were required 
to match their written diagnoses with the diagnostic categories listed 
on a checklist. This transfer, made after the students wrote their 
diagnoses, was used to establish a standardized vocabulary for analysis. 
The students did not see the checklist .while they were writing their 

* 

• diagnoses. . " ' * 



Djiff erences : and Similarities Across Preceptor Gro*ups* v , *\ 



Features 



Gronp 



Out of class 
practice on 
simulated cases 

Practice with s 
real children 

Use of Informal 
Reading Inventory 
in diagnosis ' ' 

Use of weekly 
logs for^ feedback* 

Number of vital 
signs studied 

Decision aids 
used 



Text used 
(for outside 
reading) 



no 



yes 



no 



yes« 



f lowchart c 



^Ekwall c 



yes 



yfes 



yes 



S 
no 



yes. , 



y.es 



no 



yes 



7 



diagnosis , and diagnosis 
remedxatiofl * ard remediation 
charts* 5 charts^ 



P.earson & 
Johnson** 

MORAL for ' 
preceptors 6 



Pearson & 
Johnson^. 

MORAL tor' 
preceptors* 



-^Developed by Q^orge .Sherman. 



•t 



b . 

.Developed By Deborah Cure to.n, Linda Patriar^a, and Gwyneth Stewart. 

C ' * * - 

Ekwall, 4 E. Diagnosis and remediation of the disabled r&adeSr . 
Boston, Mass.: Allyn & Bacon, 1976. * r \ 

d v 
Two texts: / * . 

Pearson, P.D., & Johnson,*' D.D. Teaching reading comprehension . 
New York: Holt, Rinehart & Wifiston, 1978.- 

Johnson, D.D., & Pearson, P.JD. Teaching reading vocabulary % 
New' York: Holt, Rinehart & Winston, 1978. 

virisonhaler, Weinshank, Cureton, & Blatt (Note 5). 

A, IV 



Ins taivt' Word Recognition 



How to measure 
Che proceaa? , 



' What la Che 
process being 
measured? 



(performance on) (acace of) r 

Sight Word* Llac ' Ins cant Word. . 

' Recognition 

*9ral Reading def. che ability 

- error analysis ' to aee and 

r say a word 
» * accurately 

v If aaked co 

do so quickly; 
4 ofcen referred 

Co sa alghc 
vocabulary 
t alghc words 



Faccora ThaC Effecc ■ ^ 
Che Proceaa 

A. Visual Diacrtminetion of Worda 

* B. Visual Hemory of Worda 

C. Seavandc repreaentations 

* (neanings) of words 



/ 



D. /Association of grapheaic 
wlch phonologic represents 
of vonA • 



itidh 



E. AssoclsClon of grapheaic 

wlch semantic representation 
of words 



Possible Problems ' 

A. Confusion, of look-alike words 

B. Inability Co remember words 
encountered In prlnC ; 

C. Word meanings Lacking 

0. Inability to pronounce a written word 

" . • - \ 

E. Inability Co attach' meaning tq^m wr^f ten 

. ' r 



•3. 



word 



* Learning Process 



Learning: change 
In one's semantic 
net induced "by 
effective practice 



Learning -Proceaa 
Effecting Factors 

\ 

Amount and condldons 
"of effective praecipe* 

4* Attention of the 
Learner 

1. -motivation to 
read 

a. Internal 

b. ♦external 



resder's self-concept? 



jjy Relevan 
of che I 



macch between 
Inceresc 'abilities 
and materials 



Relevance (tranaferabilicy) 

Che practice task Co 
the learning task , 

Learner'a correct 
'preceptlon of the 
task 

Jt 

1. viaual acuity 

2. auditory acuity 

3. clarity^ of the . • 
learning objective * 

4. clarity of che * 
criteria for aucceaa *s_ 



'D. Corrective feedback 

1 • accenclon v feedback< 
•loop 

•« *• 
2, dependence of 
feedback on 
, performance 



Possible >robjema ^ I 

Defective prsctlce 1 

t 

A* Learner's attention unfocused 

1. Lack of aoclvsdon 

0 s. no interest Co resd 

■b. envlronaenc noc conducive co * 
' • reading 

2. 'poor self-concept 

3. poor match between Interest 
macerlal* and readiness skills 

'B. Irrelevanc Cask 

/ 

A 

" •> 

C* Incorrecc j>ercepdon of che cask * 

/ 

1. poor vision 
2* poor hearing 

3. objeccives unclear # 

4, criteria unclear 



D. Defective feedback 

1, negative, feedback loop ' 



t, feedback inappropriate for the 
task 



figure 1. Diagnostic/Remedial chart *for first 

er|c * ' . Xi 



vital sign of reading. 



Affiliation Study 1980 Diagrfcst.ic 7 Remedi al -Record ' ' . • 9" - ' 

Case Name ' • \ ■ . , r, , * ' ..*.'. 

Your Name ~ -7^-*- -. . 

Date ' ~ r ■•— — — ' V . 

*. " ' ' '< . • v • ' * ' • ' - r 

Does the. student have, a prVfcAem wi*th INSTANT WORD RECOGNITION'' 

r- ; - . 

(Circle One) ■ ' Yes ' • \ No 1 > ' 

- s Pr' -- - C 

On what >basis was this deqi'srion 'made'' • 1 * ' 

* . 3 . . \ • ■ • * • : * 

• . • — - " " • . 

\ *% . . » • , '1 ^ 



If rio, the* continue wfth the. next' pr<?htem area, on paae 3. 

" •' ■' ' '.' : \ 

Tf yes, describe the important factors that have ^contributed .to this 'problem. . 
.For;each factor, suggest remedial procedures, for i ts 'imp rovement . Continue on * 
the 'next page if requ/reb.I •* . N 

1. describe one < -f actor .\ contributing tq s the problem with Iristfcnt Word' 
Recognition, . .. " <• 

' • .' ' .■ • c - > 

Suggest remedial p rocedures .fo*r at Levi ating *th,i s factor, * 



2 # 0escribe another factor contributing to the prfcblea with T r stant<* Word y 
Recognition. . „ • 

' ' ' * 1. \- ■ * 

' • . ' ' . ' ' \ ' ' < 

Suggest remedial procedures for atleviatingf this*factar. * . * * 

* «. - * * i •* 

3^De^^i*ite^ another factor* contributing to tfre problem with Instani' ^ord 
Rec*og«itian . J ^ 



Suggest remedial procedures for al levtat.ityg this factor. 



< 



Figur^ 2. Decision aid used by students ia<Siagnosing .c^ees. 



1 

■ * 




10 



. "The chepklist^flisted the sevens vitai signs a's major categories, , 
* requiritig a decis/on as *to^ adequate or £nadequat,e performance *f or each. 
Subsets under e«h vit^l sign include specifics important to the sign ^ 

* \ ' * ■ ; ' 

and decisions as to whether- those performances were adequate or inade- , 
quate. Under* each vital sign, an "'Other 11 category was listed to 
accommodate those diagnostic statemehts on the student's "diagnoses that 
could, not be translated into existing 'categories . In addition, some \ 
* learning effecting factors were listed Separately at the end of the 



\ 



H .-> < ^ 

checklisrt. (The complete checklist is contained in Appendix B*) 

. " . • .. . . > ' . * • - ^s 

At the end of the session, the written diagnosis, record form for 

' ■ *, 

cues, and checklist, of each student were collected and filed. Each 
student wds assigned a subject number using a random-order table 

* election, and all data were processed* 

* * v , * 

\ £ ' The first test session was held prior to any group meetings (pre- 

test}. Identical. procedures were followed for the" midtest (approxi- 
mately five weeks later) and the posttest ( (at the end of the* 10-week * 
'•session), Iti all these tests, 9 students diagnosed the same case, * • 

thus -.enabling a progress profile to be established. A wefek after 
, the posttest (Posttes^ 1>, another posttest was given (Posttest 2^. . In; 
9 Q test, students diagnosed a different simulated case, one which 



1* 9*» 



they hiad never seen, and, in addition, were allowed to use the written 

diagnosis and remediation charts and flowcharts, which were part of 
■ * 

their in-class training and practice on cases. In the previous test 

• X • < ' ' 

sessions, the charts were not available; thus, the students had to 
internalize the material, on the charts as a guide to writing their 
diagnoses. * ' * 
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Results 



jJ^JPhi^correlation and Porter- statistic were calculated to fine? the ^ 
inter-student ^agreement, the studenfc-preceptpr agreement, and the 
•inte^-preceptcy: ^agreement. The. latter indicated, the agreement of 
experts on each case and" served as a standard against whic;h student 
performance could be measured. (For a discussion of t^ese statistics, 



see Appendix C„) 



A member of the Clinical Studies Project checked that all transfers 
f ^ 
from the written diagnosis to the checklist were accurate, being careful, 

not to tamper with intent. Such errors as 4 marking an item as both 

* >.'•'-. 

adequate and inadequate and omitting vital signs indicated as problem 
areas on the written diagnosis were foun4 in both t Student and preceptor 
checklists and were corre(ct^d* All analysis vas then made on these < 
amended diagnoses. ^ , ' * 

■ '■' % ' '■ 

Total Diagnosis 

'The results of the total (diagnosis, which includes both vital signs 

-j . • - 

and effecting .factors, are presented iri Tablet 2. The -table cptitains 

r * ' \ ^ 

the agreement of each student with his/her prec^ef^or for all three 

groups, on all five cases. There wer^ 128 categories of statements— >^ 

64 each of adequacies and inadequacies. 



Vital Signs and Effecting Factors 

* * 

Because every diagnosis 'consists ^of two components, vital'signs 

" • • • • / 

* 

' * .. 14 ' 



labie ^ . In0 , i' rece{> tor I raining 

Agreement^ of Student With Ells/liejr Preceptor on Complete/. Corrected Dlagriobitj 0 - Phi Coefficient 

* • P - Poiter Statf&tic 



\ 

.Case 

1 


% Pretest 


* Mldtest * 
* * 


* Posttest 1 


Posttfeit 1 2 




PTG 1 
0 P 


PTG 2 
0 P 


PTG 3 
0 P 


PTG 1 
0 P 


PTG 2 
*0 P 


PIG 3 
0 P * 


PTG 1 
0 P 


PTG 2 
0 P 


PTO 3 " 
6 P 


PTC 1 
* P 


PTG 2 
0 P 


PTG 3 

D 

V L 


.62 ,.46 


.23 .19 


.38 .29 


.64 .46 


.49 .41 


.46 .36 


.49 .34 


.58 .50 


.41 .32 


.14 .14 


.17 .€0 

4f 


r* 

.29 .24 


2 ~" '* 


"TTo .26 


.38 .33 


*.26 .21 


.34 .29 
» 


f 

.50 ,44- 


.39 g.31 4 


* 

^.39 .32 


.59' .55 


.43^32^ ' 


. 30 . 2'6 


.56 .55 


.29 .24 


3 


.34 .28 


.46 * . 36 


.46 .33 ' 


I 


.48 .39 


.'21 .18 


* 

.14 .16 


:44 . 39 

*» 


^36 ,.28 


.44 .35 


737 .J5 


,12 .U 


4 


« 

.50 .35 


.38 v .26 


.41 .32 ' 
• 


.71* .59 
» * 


.33 .29 


'.50" .40 


.50 .40 


.*.3£ . Jl 

*■ 


*** 

.53 .43 


.31 .26 


.44 .39 


; ^ 

Al .26 
# 


* 

5 


-i 

/34 .27 


.27 .22 


.34 .26 


.52 .41 


.35 .27 


».^6 


.64 .52 


.35 .30 
* / 


.35 .2$ 

V 


36 1 .28 


.35 %31 


.21 .19 ' m 


Graod 
Kean 
St. Dev. 


.32 
.14 ,06 


.34 .27 . 
.09 .07 


.37 *28 
.08 -.OJf 


.48 ,".'39 
.2#* .15 


.43 .36 
.08 .08 


,37 S.iJ 
.12 .09 


,-43 -'45 
.19 .13 


.46 .41 

.12 .ir 


.42 .33 
.07 .06 


.31 .26 
.11 .08 4 


.38 .36 
,♦14 .13 


.25 ^2^ 
.08 .05 


Kean of 
Groups 




a - .38 

P - .29 




a - .43 

* P - .35 

i 


0 - .44 
P * .36 


a - .31 

P - .28 

/ 



and effecting factors, the .d ign f 9 ses w£re analyzed for'each qc 
part. Table 3 contains results' on vital ^signs only (domains 14), 
and Table 4 presents ef f^ctir.g-factors results (domain = 112)*. /Table 
5 presents inter.-s^tudent agreement for the three types of diagnoses. 

Inter-Preceptcft: Agreement • -.•«✓'' 

Table .6 presents the inter-preceptfcr ^agreement under all three 
conditions (tcrtal diagnosis, /vital signs only, and effecting factors 
only)* The highest agreement appears to be between the preceptor of 
Group 1 and the preceptor of: Group 3; the lowest agreement was between 
Preceptors 1 and 2. Preceptors 1 arid 3 were trained and practice at 

• s . . . 

■Michigan 4 §tate University. Ereceptor 2 trained at both Michigan State 
Un^persity and other universities and is currently on the staff at 

• ' " * ■ o ■ i 

another university 'The results may also show the presence of a 

« «*. , » 

ceiling effect* that is, a limit under present; analytical treatment 
to -whi disagreement can be otained. * * • 

' : * ' ■ 

Cues , ^ 

In Table .7, the student-preceptor agreement on cues selected is 

presented. Agreement appears to be lower thaa that on diagnostic 

statements; Likewise, agreement among preceptors on which cu^^to 

select was also' lower than their diagnostic agreement (Table 8)/and 

' * • t * 

.' % 

again, the best agreement was" between Preceptors 1 and 3. In addition, 

1 •* . * * 

these two preceptors agreed *more closely on the number of. cues to 



r 



■1 : 



. Agreement of Student with his/her Pref eptor* c^n Vital Sign* 



Case 



1, 



Pretest 



PTC 1 
0 P 



.71 . 75 



PTG 2 
0 * P 



.7] .75 



PTG 3 
0 P 



.43 .56 



. Midtest 



PTG 1 
0 P 



1.00 1.00 



PTG 2 
0 P 



ptg> y 

0 i_ 



.00 ,L0Q 



Posttest 1 



£XQ 1 
0% 



71 .75 



PTG^2 



,43 .56 



-*J 

PTG>3 

0 P 



. A3- ,56 



PTG ■ Preceptor /Training Group \ 
, 0 - Phi Coefficient 
P 4 - porter S a t<ttistic 

Post t-ggj- [ 1 



PTG i 



>, PTG 2, 



.29 ,*4 



'.A3 .56' 



PTG 3 

0 ' ? 



.A3 -.56 



.43 .56 



.71 .75 



.71 .75 



.A3 -.56 



.A3 .56 



-:71 ; .75 



1A . AO ' 



1.00 1.00 



•Jl -75 



.14 .40 

V 



1*00 1.00 



.]4 .40 



4 



.71 .75 
/ 



.43 .5$, 



• A3 



Grand 
Mean 



Std. Dev. 



.54 1 .64 



.15 .10 



.14 ;40 



.14 .27 



.43 .56 



43 * ,56 



.43 .,56 



>1\ .75 



-.14 .27' 



# 

.•43 .56 



.14 .40 



.43 .56 



jr. 



.71 .75 



.43 .56 



.71- .75 



.43 .56 



75 



,.43 .56 



.71 .75 



-.14 .2? 



.71 .75 



.71 .75 



\M .56 



,71 .75 



.71 .75 



,43 *.56 



1.00 1.00 * 



.65 .71 
.13 .08 



.37 ..55 



.3K .21 



.54 .67 
.43 .27 



.54 .$4 
15 ' 10 



.66 .72 

24- 18 



.37 .55 
37 t21 



.60 .69 
.25 .19 



-5- 

.48.* 



.48 .60 
.24 .15 



.46 -59 



.33 .24 



'? .56 1-.14 .?7 



.43 .56 



.71 .?5 



.69 
.25 .19 



.43 .36 



.43 .56 



.26 47 
.26 .13 



Mean of 
Groups 



0 * .'52 
P -^63 



0 



.56 

.68 



.44 

.58 



•a 



ERJC .. 



19 



1 



' « * * Table 4 

/ 'Agreement of Student vith his/her Preceptor on Effecting Factors 



PTG Preceptor Training Group 
9 - Phi 1 Coefficient 
1 P - Porter Statistic 



Case ^ 
1 


• * 

Pretest, * 


Midtast 


* . Posttest 1 


' Posttest 2 


• PTG 1 
v J p 


» rfc 2 , 

0 p - 


f PTG 3 
0 p 


^PTQ 1 
a p 


PTG 2 

M P 
^ P 


PTG 3 

0. P 


PTd 1 • , 

M P 


PTG 2 
0 p 


PTG 3 1 ' 

M P 

0 p 


PTG 1 
0 p 


PTG 2 
. p 


PTG 3 
M p 


.39 .2? 
* 


\oo ,04 


\ 

"\l$\..09 t 
* t * 


.45 .24 


. .37 .29^ 1 


' .14 .11 

-* 


*31 .16 

—4 


.56 .46 
_j t 


.31 .21 


.08 .07 


'.02 .09 


.3^ .13 


* i 

. 2 - 


IB 

.19 • .16 

\ , S 


^.22 * .1? ' 

• « * *• 


- 

.01 .05 


T^9 .22 

*> 

V 


.48 :40 


) 

• V .15 


.40 .27 
* 


.48 .44 
t 


# 

".39 .26 


.18 .15 


.51 .46' 


/20 . 16 


3 

4 F 


. 16 . k 


< 

* 

' .27 .49 


i 

.27 .14 


.18 .14 


.42 .32 
• - ) 


-.08 .00 


.16 .13 


.34 .29 


42 .15 


.25 .19 


.30 .26 


.03 .05 




-.03 .00 ' 
A. 


/ 

.29 .14 , 


< .17 .13 

* 


0 

.70 .53 


!29 .25 


.38 .26 


.40 .2* ; 


.32 * .26 


.30 .21 


.20 .16 * 


S • 

.24 .17 

i 


.27 .17 
• 


5 


fm 

* 

. .16 .13 


1 , 

0 

.24 .18 


.28 .18 


.17 .13 

/ 


.42 .29 

* 

-A 


. 14 . 12 


♦.43 .31 

r 


* 

'.40 .32 


.17 \14 - 
• 


.24 .ji 


1 

.26 .24 


$05 .08 # 


4 Praa4 
,Mean 

Std. Dev. 


.17 .13 
.45 . 08 


, .20 .15^ 
.12 .06 


; ,.i7- .12 

.11 * .05 


•.36 .25 
*22 4 16 


.40 .29 

* 

.07 .07 


.15 .13 
* 

.16'* .09 


.34 .23 
.11 .08 


.42 .35 
.10 .09 


.28 \l9 
.09 .05 


.19^ * 15 
.07 .05 


\27 .24 
.17 m .14 


.14 ."12 

j 

.10 *.05 


Mean of 
Groups 


9 - .18 
P - .13 


fy- .30 ' ( 
• ' P - AX * 

* ~ -. 


0 - .35 < 
. P - .26 

* 


0 * .20 
' P * ,17 
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Table 5 
* 

Inter-Stu4ent Correlation 



16 



Total Diagnosis 
S inula ted ' Pretest 



Mid test 



Post test 1 



Kejr C 

0 - Phi Correlation * 
P ■ Porter Statistic 



fcost test'2 



Case 




0 


P 


0 


P 


0 


■ P 


0 


P 


1 


'.26 




.30 . 


.26 , 


.37 



.33 


.41 


,.37 


2 


9Q 
. 4 7 








. *♦ j 




.41 


. 34 


3 


!37 


.25 


,31 


.27 


.40 


.34 


.38 


.31 


4 


r 


'.46 


.35 


.50 


.42 


.43 


' -.35 ( 


.35 


^32 


5 


t 


.31 


r 

.25* 


.35 


■ .29 


.47 


.39 


.48 


.42 


? 
S.D. 


•34 
' .08 


,26 
.05 


.36 
.'08 


•31 

-.06 9 


.42 
. 04 t 


.37 
.05 


.41 * 
.05 


.35 

* 

.04 



CI 



a 



Vital Signs 7 Only 

Simulated , 
C$se \ a 



Pre test 
P 



Hid test 
0 P 



Post ties 1 1 
0 P 7 



Post test2 
P 



' 1 % 


.24 


.46 


.43 


.57 


.62 


.69 


i 

.72 


.74 


2 


.81 


.83 


.43 


.57 


1.00 


N l,90 


.62 


.71 ■ 


3 


.24 


.46 


.42 


.56 


.62 


.69* 


7*2 


.70 


< 

4 


.43 


■ .57 


1.00 


1.00 


.43 


. .57 


.43 


.57 


5' 


.24 


.45 


.62 


.69 


.62 


.69* 


.81 


/ 

.83 

i 


- 'X 


.39 




.58 


" .68 


,66 


173 


.64 


.71 


*5 .D. 


.24 


.16 • 


.25 


.19 


.21 , 


4^- 






Effecting F 

Simulated 
Case 


actor's 
0 


Pre test 
• P 


P 


* 

Mid test . 
P 


0 


J I 

Post test 1 


Post 
0 „ 


j 

test 2 
p 


1 


-.04 


.02 


.21 


.18 


.26 




.32 


.29 


2 


.11 


.11 


.27 


.24 


.42 


.39 


.31 


.24 


3 


> 
.13 


.07 


.25 


.20 


.22 


* .20 


.29 


.21 


4 


-14 


.10 


.36 


.28 


.37 


.27 


.27 


.24 


5 


♦08 


.08 


.18* 


.16 


.26 


.21 


.32 


.28 


X 4 

S.D. 


.ok 

.07 


•W 

< .04 


„25 m 
.07 


.21 

; .05. 


.31 

.08 


• 2 6 

.08. 


.30 
.02 


.25 -V, 
.03 


i 

* 


r 


,* 
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Table 6 

Preceptor/Preceptor Agteement on Diagnosis 



0 » Phi Coefficient 
P * Porter Statistic 



PL = Group 1 Preceptor 
P2 = Group 2 Precepjtqij\ 
P3 = Group 3 Preceptor 



v 



°3 



Case 

...... 


Total Dx 


Vital Signs 

• ** 


Effecting Factors 

• 


rl VS» r/ 


rl VS. ij 


"DO tro 

VS, rj 


PI 1TO P9 

rl VS. r £ 


rj. vs, rj 


P9 VQ PI 


PV vq P2 


PI vs. P3 f 


P2 vs, P3 


0 P4 ? 


0 P 


0 p 


0 • P 


/ 

4 p 

* 


0 P - 




0 P ■ 


0 P 


. i 


. .29 .23 


.68 .56 


.3(7 .29 


.43 .56 


.71 .,75 


„ . 71 .75 


.14 .10 


* 


• 1? . !<♦ 


2 


.26 .24 


.51 .40 


.42 .32 


.Ik- .40 


* — — — 

.43 .56 


.71 .75. 


''.28 .21 : 


.45 .33 

* 


.34 .22 

• 


• 


• ' _ r 
.55 .43 


.82 ** .72 


. .51 "'.3a- 


...43 .56 


1.00' 1.00 


.43 .56 


.52 .37 


, .51 .36* 
-t) 


^.45 ".28 




.33 .28 


.57 .46 

;s ' • 


.57 .45 


"'.14 . .40 


.43 .56 

t 

* 


.71 .75 


. .29 • .21 


>56. .42 
< 

-* — 


.41 .27 


5 


* 

< 

• .34 .24 


.38 .29 


.32 ..26 


1.00 1.00 

/ 


.43 .56 


V 

:43 .56 


.23 .16 


.06 .07 


♦ 

.23 .16* 


Grand Mean 
Std. °t)ev. 


.35 .28 
.11 ' . 08 

t 


.59 .49 

;.i7 .16, 


'.44 .34 
.10 .08 

i B '^N. 


\ 


.43 .58 
.35 .25 


.60 .69- 
• .25 .19 


.60 .67. 
..15 .10 

* 


.29 .-21 
.14 ,.10, 


.41 130 
.20 .13 


" .32 .21 
- .11 .06* 


— 4 

Mean Across 
Preceptors 


0 » .46 






0-..54 
[' • P - .64 


/ ** 0 = .34 
' P « .24 

• 



. Table 7 



Agreement of Student With His/Her Preceptor on Cues Collected 



PTG r Preceptor Training Or- 
0 - Phi Coefficient 
P - Porter Statistic 



Case 

/ • . 


Pretest • 


Mid test 


Posttest 1 




Posttest 


2 - 


PTG I 
0 P 


PTG 2 
*0 P 


PTG 3 
0 P 


PTG 1 ^ 
✓ 

0 P 


PTG 2 
0 ' P 


PTG 3 
0 t 


PTG 1 
0- P 


PTG 2 
0 P 


PTG 3 
0 P ^ . 


PTG J 
"0 P 


PTC 2 
.0 V 


PTG 4* 
0 P 


*. 00/ . 09 


i 

.30, .35 


.08 *.13 


— c— 5 

.27 .21* 


.08 .20 


.08* .13. 


.27 .22 . 


.2B .37 


.13 ..4 




.20, .-J5\ 


.14 .16 


4 

2 


.02 .10 ^ 


' l 

.19 .28 


-.02 .09 ' 


.19 ,21 
t 


.04 .14 


.18 ^.18 


.20 .21 


* 

m .03 .19 


.13 .15. 
\ 


.13 .17 

# 


.11 .28 


-.11 .05 


* 3 


.11 .14 


9 

.11 .21 . 


-.16 '.02* 

I \ 


.70 .58 ' 
/ 


.09 .21 . 


.20 .17 


.45 .32 


.01 .20^ 


.02 .06 


.52 -41' 

• 


.04 \17 


-.02 .05 


4 


-.18 0 


.06 .17 


02 .07 
* 


.28 .24 


r 

.12 .31 


.10 .12 


.02 ..10 


.15 .28 


.20 .17 


.14 .17 

-2- 


* 

A2 .29 


.04 .10 

> 


5 


.09 .18 


-.14 .12 

* 


* 

.13 .13,} 


-i — 


-.04 ,18 


.17 .16 


.20 .22 


.16 .20 


.14 .11 


.08 .16 


.00 .21 * 
* 


.19 .17 


^Grand, 
Mean 
Sc. Dev. 


.01 .10 
«11 .07 


Hl(P\?3 
.16 ..09 


.00 .09 
.11 .05 


.29 .27 
.25 .18 


.06 .21 
'.06 .0,6 


.15 * .15 

* 

.05 .03 % 


,23 .*21 

..is .oa 

t 


.13 .25 
.11 .08 


.12 .13 

\ 

.07 .04 
» « 


.p .24 
.19 .11 


.09 .26 
.08 .07 


.05 .11 

\12 # .06 . 


Mean of 
Groups 




0 - .04 
P - .14 


* 




0 - .1? / 
P « .21 




0 - .16 
P .20 < 


P - .20 



25 



* CO 



26 . 



Table 8, 



Inter-Preceptor- Agreement on Cues Collected 



Case 



PI vs^P2 
0 P 



PI vs. P3 N 
0 P 



1 

P2 vs.P3 
if 0 P 



.08 



.12 



.30 



.24 

V 



.22 



.21 



;22' 



.22 



.50 



.30 



.24 



.02 



.09 



.28 



«*21 



.07 



.10 



H' 



_ ; .16 



.27- 



0 



.26 ' j, .18. 



, .36" < .2T\ 




Me,an Across Preceptors 



0 ■ .'23 JP - .21 



_Ke2_ 



.0 « Phi Coefficient - 
P'=, Porter Statistic 
PI ■ Group 1 Preceptor 
P2 «v Group 2 Preceptor 
P3 » Group 3'.Pre*eptor 
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collect In order -to diagnose a case/ Preceptar 1 collected 14 cufes, 

N 

Preceptor 2 collected cue^, and Preceptor 3 collected only 11.*^ 

» 

Changes. in\ Diagnostic Agreement * 

r / r 

Figures 3 - 6< show the changes in diagnostic agreement for all 

students, regardless of group attendance (N=15) , as. they progressed 

through the course. The pre- to post test changes primarily show I 

growth in both student-preceptor and inter-student agreement. 

However, for th^ second posttest, frhen students were faced with a . 

* * s • 

new case, student-preceptor agreement fell off sharply, whereas 
inter-student agreement remained constant. 

- r 

, Discussion * 

* i 

, Differences in performance among students of the three groups 

could not be related solely to their presence in different training 

* 

groups (i.e., the differences involved in their training). There were 

only five students each group, each student tested om a different 

simulated case. Differences in simulated-case difficulty and in student 

• • * ^ 

ability and background were among** the factors that could have inf lu- 

etffced performance on diagnosis more' than the details of preceptor 

training.* Therefpfe, conclusions need t$ be drawn on mean statistics 

across all three groups (15 subjects) .rather than on the limited sample* 

of s^hgl^ferpreceptor training groups. 

Certain statements can^be made about the efficacy of training. 

Results similar to that of the previous study (Gil, Polin, Vinsonhaler , ( 

&jVanRoekel, Note 3), show that agreement among students and with • 

It « . 

preceptors was aided by the use of decision aids in diagnoses in alJL 
four ,te&t sessions. Earlier observational studies (Welnshank," Nofce 1) 
involving even experienced reading cLinicians in which no such decision 

. ' • ' • . 28 „ 
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Figure 3. Changes in mean student/preceptor agreement across all 
four testing sessions (Phi Coefficient)* 



Legend 
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Figure 4. Changes in^mean 'student/preceptor agreement across all 4 
testing sessions ^(Porter Statistic). 
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"Figure 5* Changes in meaji stfudent /student agreement across all 
four testing. sessions (Phi Coefficient). 
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Figijre 6. Changes in mean student/ student agreement across all four 
, testing sessions (Porter §££tistic). 
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aids were us£d resulted in considerably lower diagnostic agreement. 
Two conclusions confirming /hope of the previous training study &re • 

(1) that the use of decisioa-^ids appears to be of value i*i obtaining 

» ■ 

higher diagnostic agreement, and (2) tflfrt student per fotmance 'does* 

improve with" training. It \Jpuld appeal that pefhaps additional 

practice with simulated cases would sljarpen-students 1 diagnostic 

skills. However, the limited inter-preceptor, agreement' seems to 

indicate that, at* ^east' under the present testing Venditions, further 

increases in diagnostic agreement may not be attainable. For example 

the preceptors diagnosed the cases only once and this diagnosis was 

'used for comparison with all* three student diagnostic sessions'! 1 ^ / 

\ 

Thus, there is rid data* on intra-preceptor agreement — how closely 

\ * 

preceptors agree with' themselves when diagnosing a case avsecond 
or third Jtime. . ^ 

Student-preceptot agreement was not maintained when ^fhe students 

« 

were given a new case to diagnose (Posttest 2). However, the inter- 
student agreement remained at tfhe same level. This may mean that 
somehow the students were using strategies. not taught^ or intended* 
by the preceptors. 'This poses the question of whether" the skills 
'obtained during training can ie maintained and generalised. Will 
classroom teachers continue to use £he organization and model in 
theit classrooms? Interviews with students at the end of the study 
elicf ted very favorable response to the training and a desire on , 
their part to incorporate their new knowledge into their classroom 

/ ' • ' 

operations. For example, analysis of the logs kept by the students 
in Group 3 showed students developing increasing confidence in their 

r 

ability, to diagnose reading difficulties cases. However, if we use 



student-preceptor agreement as the criterion, the results of Posttest 
2 do not appear to warrant such confidence. 

In summary, students do appear to gain* confidence as well as 
precision with training on a model, use of diagnostic aids, -arjfi ^ 
practice with feedback. The latter is most practically accomplished with 
simulated cases both in terms of time required and control ^^feedback. 
The goal of reaching and maintaining a high level of precision (and 
accuracy if agreement with preceptors is an indicator) poses a need" 
for more than the current 10 - 14 weeks of training now in use by most 
teacher education institutions, and, more importantly, more exposure to 
a variety of cases for practice. In addition, it would be valuable' 

to follow atudents' performance to see if gains made can be maintained / 

* 

and translated into meaningful performance in the classroom. 
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AS80/SPRING 1980 3/15/00 i 

SiJbJect Instructions for Observational Sessions 



The purpose . of the Diagnostic, Perf ormance* Session is both "to 
instruct ypu and. to evaluate your Progress as' a d j agnos liciram of 
reading disabilities* The Diagnostic Performance Session Veouires 
you to diagnose a simulated case (SIMCASE). of reading disability, 
tour SIMCASES are "simulated - only in the sense that the real , chijd^ 
is not present* The data in the SIMCASE was obtained from real 
children with reaj reading disabilities* %j 

MATERIALS The materials you need for this diagnostic performance 
session are as follows* 

<1) The SIMCASE including! I 
* ■ 

♦ The List of Available Information « ., 

♦ The Referral Information ( Initial • contact Folder) 

♦ The Bom of Materials ' 

(2) The Diagnostic / Remedial Record • r ^ 
*f3) An Audio Tape Cassette Flayer - - 

* If you do not have any of thes^g items signal your instructor*. * 
EBOCEDUBES fc • 

To review* the Procedures for the "^bfe^ervat ional session will be as 
follows; following: ' 

(1) Read the initial contact ( referral ) information* Look at the 
sketch of the child and listen to the taped interview; ' * 

(2) .Diagnpse the SIMCASE by. .collecting whatever further data yoi£/ m 
wish? 

(3) Write up your diagnosis and remediation using the Diagnostic 
/ Remedial Retord • * 

This has been a general, description of the -Diagnostic Performance 
Session* If you have any Questions? signal your instructor*^ 
* 

HOU-IO_US > E_IHE_DI&GaQSIlC--EQBtf Pick up the Diagnostic / Remedial 
Record and look it over for a few minutes* - 

The form provides a guide for- writing your diagnostic/ remedial 
Judgments* As yqu complete each page* write the case namdr your 
name and the date in the spaces provided** The*form is divided- into 
sev^n parts corresponding to important reading activities* For each 
partf state your diagnostic decision by circling* YES if the ^readifig 
activity ^is problematic and NO otherwise* Then* describe the basis 
on which the decision was* made/ Finally** 'if the area is 
• prbblematic* * describe t any factors that have contributed to the 
* problem by Dwrdting down the factor ? and 2) providing remedial 
suggestions' for the factor* 
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Subject Instructions for Observational Sessions 

1 

BOW_I0-USE_A-SIMCASE ' 

Let's, now consider the specifics of how to use a SIMCASE*' fhfe ca^e 
consists of the following items* » » / 

(1) The Initial Cantact (Referral) Information 

(2) The List of Available Information > 
<3> The Box of Materials 

The lDitial«Coptact«lDf ormatioD provides m some basic introductory 
information about the child* It" includes a referral statement* a 7 
sketch of the- child* and a taped interview* 



i 
i 



The Id£ antf3±ioD_lDyeD±ory is the list of materials collected "about 
the case* Each item of information is, described by a KEYWORD that 
refers to its title and location in the box <e*g* ? keyword for 
Durrell^fcral Reading test booklet is DUR3) ♦ , . »■ 



Now open the SltdCASE-Box.*. 

N.ote — tbeJ^KEYWOBDS^-OQ- — the f older s* Ibese«keywords-.corresE:oDd-±o 

thpse^iD^tbe^lDf ormatioD-I^DweDtory^ To find information in the 
SIMCASE — locatfe the proper . KEYWORD in the Inventory* Use this 
KEYWORD to find the proper folder in the box* 

As in the case of DUR3* find the large ' folder entitled DURRELL* 
'Within; that folder are smaller colored . folders in which will be T 
found the specific item DUR3* Only remove the desired information? 
not the colored folder in which it is located* "\ 

Now ttiat you have practiced* ybu can begin to diagnose and remediate 
your SIMCASE* 

Begin by opening the Initial Contact folde^* Examiftfe the picture 
and the information* and listen to the tape* * / """""" 

When you have finished with the Initial Contact Material* 'signal 
your instructor* After he of she responds you may 'begin collecting 
information to diagnose* your case* You will have .45 minutes* in 
whiclv to collect information* The proctor will tell you when yoMf* 
time is up* r * # 

You will have, an additional 30 minutes in 'which to* fill out the 
Diagnostic / Remedial | Record and 15 minutes to transfer the 
diagnosis tb a Diagnostic Checklist described later* 

Ele3se_da~DQ±~reftu£D-3Qy-.o£ -tbe„materials^±a^±be_ba^l_Leay^e_tbem.-OQ 
tbe-iabJLe^ # . 



If yqu have Questions or problems at any timer signal your ^ 
instructor* Relax* There are no grades to be given in this Study* 
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Ctteo Name v ; ' 

your ,Ua;ne ^ 2 ZIX 

Bi> "— — — * V../* -"•«■•'•' ^'v.y. ' . 

aWbc purpose, of the Diagnostic cb.cckl.ist "is^ W translate jflouf" "diagnosis as 
writtcr 

down. 

translation of tne ones you included in your written diagnosis. 

' * 1 • * * 

Examine, "youj written diagnosis ad recorded oh .the Diagnostic/ RemediM Record*. 
For each factor on the Record that describes a FACTOR contributing to a 
problem, do th,e .following; * * l & * 0 

. 9 t ' . " • / 

^£)Circle >thc factor on the Diagnos-tic*/ Remedial Record 
2)Huttbcr the, factor, starting <at 1 for* the first one and going up from 
th'pp<? for each -succcoiv>>^f actor . m 
* 3jLqc<*te the statement on this checklist ■ that <fnost accurately characterizes 

the faptor. i V* t . . 

- 4) PlaceTthe. factor number next to the statement on thp 'checklist* ^ 
• 5).Contij?u'e for all factors on yonY Diagnostic/Remedial Record A 

^ An* example, of ' tri(y^ process is given 'beiqw^ " \ m °< % 



jThe EiacjQOstic / Rejnedia^l Record is: 1 . > 
J 

1 
i 




1. Describe one factor contr ibutinq to the- problem with 'Instant 
Word Recognition. r<?^?~^r~ — rr^r- ; — * *" ~~ 



' I Suggest renedial procedures- for alle\^Latinrj 



3^fi^ 



^|ctor. 



■J- / ' 2^Descr»be pother factor contributing to the^ptoblen witmlnstant, 

! -A 'Woj^Recc^^i^on.^^^ ^^^^^ \ " • 

» ' * \ v — - i . — 1*» / 

♦I Suggest remedial procedures for alleviating tliis factor, - - . % . 

i . .etc; > • ^ . / - <r • : 

I$nd the Diagnostic 4 Checklist is: » * t 0 ; ^ * / J ; ■ ^ J 

. * # . . . / 

1 . ; Ins t tant Mojcd Recognition Adequate * I 

„ 2 Instant Word ls^scognition Inadequate * A 'I* 

3 . " %i Basic* Sight Words Adequate * * °V I 

4 % y ' . B.a^ic Sight v;o*rds* Inadequate % ^ . - • * .1 



f IP ^ , - Visual Discrimination 'inadequate 



^ " - j 



etc. • • ,'■ ^ ' • <C - 

"ERjc—^rTr-- - — — 7 — >- 
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1 + InfstftiA Word Recognition Adequate 

2 ^Xnsl'int Won! Recognition Inadequate 

3 dTcTrvi^c Sight Words Adequate 

4 y3asic~Sic]lit Words Inadequate 

Sight Wordc^ Learned Via Decoding Adequate* v 

6 Sight Wo r<» Learned Via Decoding Inadequate 

7 I , , Experiential Sigh£ Words Adequate , • 

6 0 ^ ffix)?cri6ntiaX Sight V.'ocds inarkjquate •/ 

, 9 ^ Wsual Discrimination Adequate - ? ^ m 

lfl Visual 'Discrimination Inadequate 1^ ^ 

11 , Visual Memory Adequate * ' • • |$ 

12 4 | Vi sual Memory ^nadeqaate 

13 ^ / Print-Hcaning Association Adequate 

14 : * Print-Meaning Association Inadequate 

1 *j ' Print-Sounn Association Adequate 

16 m PrinfcnScund Association Inadequate & 

17] Other Adequate' ■# - 

'lft ; * Other Inadequate . * Jjk 

19 Decoded Word Recognition Adeguate ■* * , * 

% 20 __DecQe3ed'*Vord Recognition Ifiall equate .* 

21 «- ^Sound-Syriihol Association - Consonants Aclcquate 

22 / Sound-Syj^bcl Association - Consonants Inadequate - 

23_. * , Sound-Symbol Association - Blemds/Diagraphs Adequate** 

.24 Sound-Symbol, Association - JSlends/Diagraphs 'inadequate > 

• 25 Sound-Symbol Ass@cia.tion - Vowels/Vov/el Patter^- Adequate 

26 », Sound.- Syi.ibol Association - Vowels/ Vowel Patterns Tnadu.Jua^e 

27 Visual Segmentation i'nto Syllables Adequate 

ffi_ 9 Visual Segmentation into Syllables Inadequate \ 

29 ' Auditory Seonontat'ifrn into Syllables Adequate, 

* » Auditory Segmentation in^> Syllables Inadequate - ^/ 

31 Blending of Sounds Adequate . , • ■ 

32 \ _ Blending <,of* Sounds Inadequate' , • 

33 * w \ Adjustment of Blended Sounds to Language Adequate 

34__. Ad justnent -of 'Blended Sounds to Language Inadequate* 

35j Use* of Root Word Adequate 

36 Use of Root Word Inadequate 

37 Use of Prefixes Adequate. 

33 ^ f Use of Prefixes . Inadequate 

3 9 k Use of Suf Ci>:es" Adequate * 

*40 . ' * -Use of Suffixes Inadequate 

.4 3 __- //Ns ^ Auditory Memory Adequate ' w ? 

42 Auditory Memory Inadequate 

43 Audi tory^ Discrimination Adequate 

44] ^ Auditory Discrimination Inatlcqua te* 

• Visual >!^u».ory Adequate .* * 

46j2 Visual Memory Inad*qu£ite * 

47_ VisucM Miscriminat ion Adequate 

43 ' Visual Dj ^criwinVrtTtpn. Inadequate > 

4^r^_ 1 Other A(!".|iiato 

Other Inadequate / ' s 

Lnq Vocabulary Ail equate . 
Lag Vocabulary Ihadeqimte 
Nmnhor of wordn Advqdnt'e 

54 ^ 'Nuhiher of words Inadequate t * ' 

55~ t Accuracy and Brondth of Do Cini tionr. Adequate 

56 a w Accuracy arid Breadth of 'Definition'* Inadoquate 

^5^7 _ /Other Admniate 




"'ATpliQatLon Jftudy 19tt0 Diagnostic Checklist 



Case Wame 
your Name 
Date 



Oral Rending Adequate 
Oral Reading Inadequate 

' Instant Word Recoqnition -Adequate 
Instant Word Recoqnition Inadequate 
Decoded Uord Recognition Adequate 
Decoded Word Recognition Inadequate 
Rate Adequate 
Rate* Inadequate 
'Phrasing Adequate 
Phrqs ing Inadequate \^ 
Intonation Adequate 

Intonation Inadequate ' 
Use of Syntax Adequate 
Use of Syntax Inadequate 
Use of Semantics Adequate 
Use of Semantics Inadequate 
Other Adequate ^ 

Other Inadequate -^s 
Reading Comprehension Adequate 
Reading Comprehension Inadequate 
Main icjea Adequate 
M$in idea Inadequate 
Paraphrasing Adequate 
Paraphrasing Inadequate 
Seq.uence Adequate 
Sequence Inadequate, 
Causc^Ef feet Reasoning Adequate* 
Cause-Effect Reasoning Inadequate 
Recall of Facts, and Details Adequate 
Recali oj£ Facts and Details Inadequate 
e£* Adequate, 
er Inadequate 
Listening Comprehension Adequate 
listening Comprehension Inadequate, 
Main idea Adequate 





94 Main idea Inadequate 

95 ; Paraphrasing Adequate 

9jG Paraphrasing Inadequai 

.97 Sequence Adequate * 

90 f Sequence Inadequate . ^ 

99_ Cause-Effect Reasoning Adequate 

19G Cause-Ef fe'ct** Reasoning Inadequate 

101 Recall of Facts and Details Adequate- 

102 Recall of Facts anc? Details Inadequate 

103 Other. Adequate 

104 ^ Other ' Inadequate 

105 . ^Atutentidn/MotivatiotV Adequate 

1^6 • At tenJL*ioii/?Iotivation Inadequate ; 

107 " Initiating Attention 'Adequate 

* Initiating Attention ^Inadequate 
Maintaining Attention Adequate 
Maihtaiarng Attention Inadequate • 
Other Adequate / 
bthor Inadequate ^ 43 
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- Case Hainc - : 

'Your Name - 

Date 



^ — 



113 l < - lectors Effecting Lcafriinq Adequate ■ y * 

114 Factors Effecting Learnihq Inadequate * 

115 Amount of Practice AcTi&uate 

116 Amount of* Practice Inadequate ^ 

117^ Attention AAdequate " 

110 v Attention Imulcquate * * 

119 Motivation Adequate 

120 \ Motivation Inadequate \ ' : 

* 1 21 Visual .Acuity Adequate « \ 

. 122 ' Visual Acuity Inadequate . % V 

123 Auditory Adbity Adequate % - 

124 Auditory Acuity Inadcqua telw . ' 

125 ~ Tranofcfj^ of Isolated Skills to fcontcxtual Materials Adequate- 
120^ Transfer of isolated Gkills to .Contextua 1 Materials Inadequate 
127^ Other Adequate * * . jSfcy*. 1 

120 . Other Inadecibate, ''^^m;^ 
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Appendix C 



Explanation of Phi Correlation and Porter Statistic 




Inter-Clinician Correlation 

\ _ . i 

Phi * 

Given a domain for diagnoses/remedia4:i6ns/or cu£s (DX/RX or CX) 

i 

for a given case, the Phi correlation is a measure of inter-cliriician 
agreement. One Pttl correlation is computed for each pair of clinicians. 

The' DX/RX/CX categories mentioned by one clinician are compared 
with those mentioned by a second clinician for the same case. This 
comparison is summarized in the table below. 



CLINICIAN B, 
SIMCASE Y 



CLINICIAN A, SIMCASE Y 
PRESENT (+) • ABSENT (-) 



-P 
R 
E 
S 
E 
N 
T 



A 
B 
S 
E 
N 
T 



(-) 



Frequency count of 
statements in the 
domain present in 
> both clinicians , 
DX/RX or CX 

9 

a 

/ 


Frequency count of 
statements in she 
domain present in 
■ clinician B f s • 
session but not in 
clinician A f s 
DX/-RX/ or CX - 

b 


Frequency count of «. 
statements in the 
domain present in ' 
clinician A/s session - 
but not fh B's DX/RX 
or CX , > 

c 


Frequency count of 
statements in' the 
domain absent in 
> both clinicians' 
DX/RX or CX 

d 
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,The calculation of the Phi correlation is 
as follows: 4 


derived from the table 


v: 1 

V 


• t * 

Clinician. A* Simcase Y ' 


CliMxian B, 


\ a(H-f) 




Simcase Y 

4 


, • ^ c(-+) 


d(-) 


% 

t * 







Phi = 



(axd - bxc) 



a+c 



(a+c)x(b+d)x(c+d)x(a+b) 



b+d 



a+b ^ 



c+d 



The statistic is bounded by -1 (statements are in cells *b and c only) 
and 1 (statements are in cells a and d only), only if the distribution in 
the marginals is equal. In all other cases the maximum and minimum value's* 
will be less than 1 and greater than -1, ■ 

An example of a' completed table is as follows: r % 



Statements ofj 
Clinician A* 
Simcase* Y 

SI 

■ S2 
S3 



Statements of 
Clinician B, 
Simcase Y * 

SI * 

S2 
S7 



Domain of 
Statements 



SI 
S2 
S3 
S4 
S5 

s2 

S7 



47 



m 



40 



Clinician A 



~Clir#cian B 



2 


i 


1 


y 




3 

* 


% 


» 






. "4 





Porter 



1 . 



Another statistic which is useful and based upon the 4-fold contingency 
table ia^the Porter Coefficient (£)* This coefficient's a measure of the ># 
agreement between two individuals' diagnoses or two diagnoses by the same 
individual as a function of the proportion of that agreement compared to the 

sum of the total agreements and non-agreements (i.e., the ++,4-*-rf boxes, and 

* * * * 

excluding the — one.) * 

If A = no. categories common to both subjects or encounters, 1 

V ■ 

B & C = no. categories mentioned by one subject (encounter) and 
not the other, then 



P = 



A r 



A + B" + C . 



This does not reflect agreement on not to select a category (diagnosis, cue, 
treatment) as is the case ^ith the Phi correlation. 

For the previous example, the Potter Coefficient would also be .67 
as the total domain of categories is pnly 5. . If the domain is large and the 
number of* categories mentioned *by both or either subject is small, the Phi 
-and Porter values' will differ, ami the Potter coefficient will appear to be 
more reliable. * % 



*ddveloped by Dr. Andrew Porter, Institute for Research on Teaching, Michigan 
.State University* * ' , , 

v >48 . * ' 



